Image transmission apparatus, image transmission method and image transmission program product

ABSTRACT

A cellular phone includes, for the sake of privacy protection, a camera for outputting a pick up image in which an operator is photographed, a touch panel to accept designation of at least a partial region of the pick up image, and a control portion to determine a masked region to mask at least a part of the pick up image based on the accepted region, generate a composite image in which the masked region of the pick up image is converted to another image, and transmit the composite image.

This application is based on Japanese Patent Application No. 2007-086711filed with Japan Patent Office on Mar. 29, 2007, the entire content ofwhich is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image transmission apparatus, animage transmission method, and an image transmission program product,and particularly relates to a portable image transmission apparatus, apick up image transmission method executed by the image transmissionapparatus, and a pick up image transmission program product executed bythe image transmission apparatus.

2. Description of the Related Art

In recent years, cellular phones capable of communication by picturephone have been becoming widely available. With picture phone, an imageof a speaker is transmitted simultaneously with speech, and therefore,it is possible to shorten psychological distance between speakers. Inaddition, portability of cellular phones eliminates restriction on placeof calling. Therefore, background images behind speakers photographed bya camera are different in each case depending on time and place ofcalling. For example, when a picture phone receives a calling while oneis walking on a road, faces of others may be included in the background,and thus, in some cases transmission of images including such abackground to a person on the other end is not desirable. In this case,one should either give up using the picture phone or move to some otherplace where the background is fair if photographed, and then resumecalling on picture phone. This results in such problems that one isunable to respond to requests of a person on the other end or it takesmuch time before answering an incoming call.

Conventionally, such a picture phone apparatus is known that recognizesthe boundary between the person and background of a dynamic imagephotographed by the camera to extract the background region, replacesthe background image portion of the dynamic image photographed by thecamera by a background image stored in advance, and then transmits thedynamic image data.

However, this requires the sophisticated image processing of extractingthe background region by recognizing the boundary between the person andbackground of a dynamic image photographed by the camera, and thuscauses problems including failure of processing if the boundary betweenthe person and background is recognized incorrectly. This also requiresa substantial amount of time for the extraction processing of thebackground region, resulting in problems including elongated timerequired before transmission of the dynamic image.

SUMMARY OF THE INVENTION

The present invention is made to solve above-mentioned problems, and oneof the objects of the present invention is to present an imagetransmission apparatus capable of privacy protection.

Another object of the present invention is to present an imagetransmission method and an image transmission program product that arecapable of privacy protection.

In order to achieve the aforementioned objects, according to an aspectof the present invention, an image transmission apparatus includes: animaging portion to output a pick up image of a shooting object; a regiondesignation portion to accept designation of at least a partial regionof the pick up image; a masked region determining portion to determine amasked region for masking at least a part of the pick up image based onthe accepted region; a composite image generation portion to generate acomposite image in which the masked region of the pick up image isconverted to another image; and a transmission portion to transmit thecomposite image.

According to another aspect of the present invention, an imagetransmission method includes the steps of: acquiring a pick up image ofa shooting object; accepting designation of at least a partial region ofthe pick up image; determining a masked region for masking at least apart of the pick up image based on the accepted region; generating thecomposite image in which the masked region of the pick up image isconverted to another image; and transmitting the composite image.

According to further another aspect of the present invention, the imagetransmission program product causes a computer to execute the steps of:acquiring a pick up image of a shooting object; accepting designation ofat least a partial region of the pick up image; determining a maskedregion for masking at least a part of the pick up image based on theaccepted region; generating a composite image in which the masked regionof the pick up image is converted to another image; and transmitting thecomposite image.

The foregoing and other objects, features, aspects and advantages of thepresent invention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view showing external appearance of a cellularphone in one embodiment of the present invention.

FIGS. 2A and 2B are plan views of the cellular phone.

FIG. 3 is a diagram showing arrangement of indication keys.

FIG. 4 is a functional block diagram showing an example of an outline offunctions of the cellular phone.

FIG. 5 is a functional block diagram showing an example of an outline offunctions of a control portion.

FIG. 6 is a diagram showing an example of a pick up image.

FIG. 7 is a first diagram showing an example of a masked region.

FIG. 8 is a diagram showing an example of a non-masked region.

FIG. 9 is a second diagram showing an example of a masked region.

FIG. 10 is a functional block diagram showing a function of determininga masked region using a pattern image of a masked region determiningportion.

FIG. 11 is a diagram showing an example of the pattern image.

FIG. 12 is a diagram showing an image in which a masked region of thepattern image is superimposed onto a pick up image.

FIG. 13 is a diagram showing a masked region after expansion orreduction of the pattern image.

FIG. 14 is a first diagram showing an example of a masked region aftermovement.

FIG. 15 is a second diagram showing an example of a masked region aftermovement.

FIG. 16 is a flowchart showing an example of flow of incoming callprocessing.

FIG. 17 is a flowchart showing an example of flow of masked regiondetermining processing.

FIG. 18 is a flowchart showing an example of flow of follow-upprocessing.

FIG. 19 is a flowchart showing an example of flow of calling processing.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will now be described withreference to the drawings. In the following description, like referencecharacters denote like parts, which have like names and functions, andtherefore, detailed description thereof will not be repeated.

The following description explains a cellular phone 1 as one example ofa picture phone as the image transmission apparatus. FIG. 1 is aperspective view showing external appearance of a cellular phone in oneembodiment of the present invention. Referring to FIG. 1, cellular phone1 includes a display side portion 2 and an operation side portion 3. Therear surface of display side portion 2 faces the surface of operationside portion 3 so that display side portion 2 is slidable in thelongitudinal direction. FIG. 1 shows a state where display side portion2 is slid with regard to operation side portion 3 to open cellular phone1. When cellular phone 1 is closed, the rear surface of display sideportion 2 and the surface of operation side portion 3 are overlapped.

FIG. 2A is a plain view of cellular phone 1 in closed state. FIG. 2B isa plain view of cellular phone 1 in open state. Referring to FIG. 2A andFIG. 2B, on the surface of display side portion 2, there are arranged acamera 24, a liquid crystal display device (LCD) 16, a touch panel 25A,an operation key 14 including a talk key 14B, an indication key 15, anda talk end key 14C, and a speaker 11. Touch panel 25A is made of atransparent material and is provided onto the entire display surface ofLCD16. An example where cellular phone 1 includes LCD16 is shown here,an organic electro luminescence display or the like may be used insteadof LCD 16.

When cellular phone 1 is open, a part of the surface of operation sideportion 3 is exposed. On the exposed surface of operation side portion3, there are arranged a numerical keypad 14A comprising a part ofoperation key 14 and a microphone 13. Further, a second speaker 12 (seeFIG. 4) is arranged to the side face of operation side portion 3.

FIG. 3 is a diagram showing arrangement of the indication keys.Referring to FIG. 3, indication key 15 includes a determining key 5, anupward arrow key 6, a downward arrow key 7, a left arrow key 8, and aright arrow key 9. Determining key 5, upward arrow key 6, downward arrowkey 7, left arrow key 8, and right arrow key 9 are button switches whichare turned ON when pressed. Cellular phone 1 is able to display apointer on LCD 16, and when operation of indication key 15 by a user isaccepted, cellular phone 1 moves the pointer up, down, right, or leftdepending on which of upward arrow key 6, downward arrow key 7, leftarrow key 8, and right arrow key 9 is pressed. This enables it todesignate any position of the display surface of LCD 16.

FIG. 4 is a functional block diagram showing an example of an outline offunctions of cellular phone 1. Referring to FIG. 4, cellular phone 1includes a control portion 21 to control the entirety of cellular phone1, a wireless circuit 22 connected to an antenna 22A, a codec portion 28to process audio data, a microphone 13 and a first speaker 11 eachconnected to codec portion 28, a tuner 23 connected to an antenna 23A,an AV decoder 29, camera 24, an operation portion 25 to accept input ofuser operation, a display control portion 30 to control display ofLCD16, a RAM (Random Access Memory) 31 used as a working area of controlportion 21, a flash ROM (Flash Read Only Memory) 32 to store programs tobe executed by control portion 21, a vibration portion 26, a cardinterface (I/F) 27, and an external communication connection terminal 33which serves as a serial interface.

Wireless circuit 22 accepts input of a wireless signal received byantenna 22A and outputs to codec portion 28 an audio signal resultingfrom demodulation of the wireless signal. Further, wireless circuit 22accepts an audio signal input from codec portion 28 and outputs toantenna 22A a wireless signal resulting from modulation of the audiosignal. Codec portion 28 demodulates the audio signal input fromwireless circuit 22, converts digital decoded audio signal to an analogaudio signal, amplifies it, and outputs it to first speaker 11. Further,codec portion 28 accepts an analog audio signal input from microphone13, converts the audio signal to a digital signal, encodes it, andoutputs the encoded audio signal to wireless circuit 22.

Tuner 23 is connected to antenna 23A and accepts input of a broadcastsignal received by antenna 23A. The broadcast signal is a high-frequencydigital modulation signal including image data and audio data. Tuner 23takes out a specific frequency signal from a high-frequency digitalmodulation signal input from antenna 23A. Further, tuner 23 is providedwith an reverse interleaved circuit and an error correction circuit, anddemodulates obtained high-frequency digital modulation signal of thespecific frequency and outputs coded data to AV decoder 29. AV decoder29 is provided with a video decoder and an audio decoder, and decodesthe coded data input from tuner 23 to generate an image signal and anaudio signal, outputs the image signal to display control portion 30,executes D/A (digital/analog) conversion of the audio signal, and feedsit to second speaker 12. While an example where a broadcast wave ofdigital television broadcast is received and reproduced is shown here, abroadcast wave of analog television broadcast may be received andreproduced. Further, instead of television broadcast or in addition totelevision broadcast, a broadcast of radio broadcast may be received andreproduced.

Display control portion 30 is controlled by control portion 21 andcauses LCD 16 to display an image by controlling LCD 16 according to aninstruction input from control portion 21. The image to be displayed onLCD 16 includes dynamic images and still images.

Camera 24 includes a lens and a photoelectric conversion element such asa CMOS (Complementary Metal Oxide Semiconductor) sensor or the like,forms an image of light condensed by the lens to the CMOS sensor, whilethe CMOS sensor photoelectrically converts the received light andoutputs image data to control portion 21. Camera 24 is controlled bycontrol portion 21, starts picking up an image according to aninstruction from control portion 21, and outputs obtained still imagedata or dynamic image data to control portion 21. Camera 24 includes animage processing circuit to execute image processing to improve theimage quality of photoelectrically converted image data, and an A/Dconversion circuit to convert image data from analog data to digitaldata. Control portion 21 outputs still image data or dynamic image dataoutput by camera 24 to display control portion 30 and causes LCD 16 todisplay the data. Further, it is possible for control portion 21 toencode still image data or dynamic image data and to store the data inflash ROM 32 or a memory card 27A mounted to a card I/F 27. Further,when cellular phone 1 functions as the picture phone, camera 24 encodesdynamic image data by a predetermined compression coding system andoutputs it to control portion 21.

Operation portion 25 includes operation key 14 and touch panel 25A.Operation key 14 and touch panel 25A accept input of an operation by theuser and outputs the accepted operation to control portion 21.

Removable memory card 27A is mounted to card I/F 27. Examples of memorycard 27A includes CompactFlash, SmartMedia (registered trademark), SD(Secure Digital)/MiniSD/MicroSD memory card, memory stick, MMC(Multimedia Card), and xD picture card.

Control portion 21 can access memory card 27A via card I/F 27. While aprogram to be executed by control portion 21 is stored in flash ROM 32in advance in this example, the program may be stored in memory card 27Aand then read from memory card 27A to be executed by control portion 21.Recording medium for storing the program is not limited to memory card27A; other examples include flexible disk, cassette tape, optical disk(CD-ROM (Compact Disc-ROM)/MO (Magnetic Optical Disc)/MD (Mini Disc)/DVD(Digital Versatile Disc)), optical card, mask ROM, and semiconductormemory such as EPROM. Further, cellular phone 1 may be connected to theInternet via wireless circuit 22, and download the program from acomputer connected to the Internet to be executed by control portion 21.The program as used herein includes, in addition to programs executabledirectly by control portion 21, a source program, a compressed program,and an encrypted program.

FIG. 5 is a functional block diagram showing an example of an outline offunctions of control portion 21. Referring to FIG. 5, control portion 21includes a pick up image acquisition portion 51 to acquire a pick upimage, a position acceptance portion 57 to accept designation of anyposition in the pick up image, a masked region determining portion 59 todetermine a masked region, a composite image generation portion 53 togenerate a composite image in which the masked region of the pick upimage is replaced by another image, a transmission portion 55 totransmit the composite image, a feature extraction portion 61 to extractfrom the pick up image a feature included therein, and a masked regionmovement portion 63 to move the masked region with regard to the pick upimage according to movement of the feature in the pick up image.

Pick up image acquisition portion 51 accepts input of a pick up image ofa shooting object that is picked up and output by camera 24. Pick upimage acquisition portion 51 acquires the input pick up image. Pick upimage acquisition portion 51 outputs the acquired pick up image tocomposite image generation portion 53 and feature extraction portion 61.The pick up image being input from camera 24 includes still images anddynamic images. An example where a dynamic image is input as the pick upimage will be explained here. When the pick up image is input from pickup image acquisition portion 51, composite image generation portion 53outputs the pick up image to display control portion 30 and causes LCD16 to display it.

Position acceptance portion 57 accepts designation of any position inthe pick up image. Position acceptance portion 57 accepts a position inthe pick up image designated by the user on touch panel 25A. Positionacceptance portion 57 identifies a position in the pick up imagedisplayed on LCD 16 from the position detected by touch panel 25A.Further, position acceptance portion 57 accepts a position of a pointerat the time when the user press determining key 5 after moving thepointer displayed on LCD 16 by manipulating indication key 15. Positionacceptance portion 57 accepts the position in the pick up imagedisplayed on LCD 16 from the position of the pointer displayed on LCD16. Position acceptance portion 57 outputs the accepted position in thepick up image to masked region determining portion 59.

Masked region determining portion 59 determines a masked region in thepick up image based on the position in the pick up image input fromposition acceptance portion 57 and outputs the determined masked regionto masked region movement portion 63. Masked region determining portion59 shifts the mode to either masked region designation mode ornon-masked region designation mode. In the masked region designationmode, masked region determining portion 59 determines, as the maskedregion, the point in the pick up image input from position acceptanceportion 57 or a predetermined range around the position. When the usertraces the touch panel by his/her fingers, or describes a tip byindication key 15, a plurality of neighboring positions are inputcontinuously from position acceptance portion 57. Masked regiondetermining portion 59 determines, as the masked region, a predeterminedregion from the plurality of positions input continuously.Alternatively, a region delimited by the plurality of positions inputcontinuously may be determined as the masked region.

In the non-masked region designation mode, masked region determiningportion 59 determines, as the non-masked region, a predetermined rangefrom the position in the pick up image input from position acceptanceportion 57. Masked region determining portion 59 determines, as thenon-masked region, a region of a predetermined range from a plurality ofpositions input continuously from position acceptance portion 57.Alternatively, a region delimited by the plurality of positions inputcontinuously may be considered as the non-masked region. Masked regiondetermining portion 59 determines, as the masked region, the regionother than the non-masked region in the pick up image.

Now, designation of a masked region will be specifically explained. FIG.6 is a diagram showing an example of a pick up image. Referring to FIG.6, a pick up image 100 includes an image 101 of an operator of cellularphone 1 as the shooting object, and an image 103 of an advertisingdisplay on which characters of “□ □ Company” are represented. FIG. 7 isa first diagram showing an example of the masked region. When, whilepick up image 100 shown in FIG. 6 is displayed on LCD 16, the operatorgives an instruction to paint out all regions other than own image 101on touch panel 25A, masked region 110 shown by hatching in FIG. 7 isidentified. FIG. 8 is a diagram showing an example of the non-maskedregion. When, while pick up image 100 shown in FIG. 6 is displayed onLCD 16, the operator gives an instruction to paint out own image 101 ontouch panel 25A, non-masked region 111 shown by hatching in FIG. 7 isidentified. FIG. 9 is a second diagram showing an example of the maskedregion. When, while pick up image 100 shown in FIG. 6 is displayed onLCD 16, the operator gives an instruction to paint out the region ofimage 103 of the advertising display on which characters of “□ □Company” are represented on touch panel 25A, masked region 113 shown byhatching in FIG. 9 is identified.

Referring back to FIG. 5, masked region determining portion 59 candetermine a masked region based on a pattern image stored in advance inflash ROM 32. FIG. 10 is a functional block diagram showing the functionof masked region determining portion 59 to determine the masked regionusing a pattern image. Referring to FIG. 10, masked region determiningportion 59 includes a pattern image acquisition portion 71 to acquire apattern image, an expansion instruction acceptance portion 73 to acceptan expansion instruction, an expansion portion 75 for expand the maskedregion based on the expansion instruction, a reduction instructionacceptance portion 77 to accept a reduction instruction, and a reductionportion 79 to reduce the masked region based on reduction instruction.

Pattern image acquisition portion 71 reads the pattern image stored inadvance in flash ROM 32. A plurality of pattern images may be stored inadvance in flash ROM 32 so that any one of the pattern images isselected based on a feature of the image included in the pick up image,or the operator may select one of the images. Pattern image acquisitionportion 71 outputs the read pattern image to display control portion 30,causes LCD 16 to display an image in which a pattern image issuperimposed onto the displayed pick up image, and outputs the image toexpansion instruction acceptance portion 73 and reduction instructionacceptance portion 77.

When a pattern image is input from pattern image acquisition portion 71,expansion instruction acceptance portion 73 sets the pattern image asthe masked region. Expansion instruction acceptance portion 73 acceptsinput of the masked region from expansion portion 75 or reductionportion 79. Further, expansion instruction acceptance portion 73 acceptsan expansion instruction from touch panel 25A or indication key 15.Expansion instruction acceptance portion 73 accepts the expansioninstruction when the user, after having indicated the position of themasked region on touch panel 25A, continuously indicates a positionother than the masked region. The expansion instruction includes theposition other than the masked region. Upon acceptance of the expansioninstruction, expansion instruction acceptance portion 73 outputs theposition other than the masked region included in the accepted expansioninstruction and the masked region at this time to expansion portion 75.

Expansion portion 75 expands the masked region accepted from expansioninstruction acceptance portion 73 to the position other than the maskedregion. Expansion portion 75 outputs the expanded masked region toexpansion instruction acceptance portion 73 and reduction instructionacceptance portion 77.

When a pattern image is input from pattern image acquisition portion 71,reduction instruction acceptance portion 77 sets the pattern image asthe masked region. Further, reduction instruction acceptance portion 77accepts input of the masked region from expansion portion 75 orreduction portion 79. Furthermore, reduction instruction acceptanceportion 77 accepts a reduction instruction from touch panel 25A orindication key 15. Reduction instruction acceptance portion 77 acceptsthe reduction instruction when the user, after having indicated theposition other than the masked region on touch panel 25A, continuouslyindicates the position of the masked region. The reduction instructionincludes the position of the masked region. Upon acceptance of thereduction instruction, reduction instruction acceptance portion 77outputs the position of the masked region included in the acceptedreduction instruction and the masked region at this time to reductionportion 79.

Reduction portion 79 reduces the masked region accepted from reductioninstruction acceptance portion 77 to the position of the masked region.Reduction portion 79 outputs the reduced masked region to expansioninstruction acceptance portion 73 and reduction instruction acceptanceportion 77.

A specific example of the case where a masked region is determined usinga pattern image will be explained. FIG. 11 is a diagram showing anexample of the pattern image. Referring to FIG. 11, the pattern imageincludes a pattern 121 that is intended to be a masked region. In thefigure, pattern 121 is shown by hatching. The pattern image includes aregion other than a circular masked region positioned roughly in thecenter of the diagram. FIG. 12 is a diagram showing an image where themasked region of the pattern image is superimposed onto a pick up image.Referring to FIG. 12, when the user indicates on touch panel 25A aposition shown by point 123 and moves the user's finger in the directionof an arrow extending from point 123, then the reduction instruction isaccepted. When, while the pattern image provided onto the pick up imageis displayed on LDC 16, the user indicates a position shown by point 125on touch panel 25A and moves the user's finger in the direction of anarrow extending from point 125, then the expansion instruction isaccepted. FIG. 13 is a diagram showing the masked region after expansionor reduction of the pattern image.

In this way, since the masked region can be changed by expanding orreducing the masked region included in the pattern image on touch panel25A, a masked region adjusted to a pick up image can be determined withease.

Returning to FIG. 5, feature extraction portion 61 extracts a featurefrom the pick up image input from pick up image acquisition portion 51.The feature is included in a characteristic shape, color, or pattern,such as human eye and characters, included in the pick up image. Thefeature can be determined by binarizing the pick up image and extractinga predetermined shape or by extracting a predetermined color portionfrom the pick up image.

Feature extraction portion 61 outputs the extracted feature to maskedregion movement portion 63. When the positions of the identical featureextracted from two pick up images input from pick up image acquisitionportion 51 at different times are different, feature extraction portion61 calculates a movement vector and outputs it to masked region movementportion 63.

Masked region movement portion 63 outputs the masked region input frommasked region determining portion 59 to composite image generationportion 53. Further, when a movement vector is input from featureextraction portion 61, masked region movement portion 63 moves themasked region in the pick up image in the direction and the distanceidentified by the movement vector, and outputs the moved masked regionto composite image generation portion 53.

Composite image generation portion 53 generates a composite image inwhich, of the pick up image input from pick up image acquisition portion51, a masked region input from masked region movement portion 63 isreplaced by a preset image. The preset image is stored in advance inflash ROM 32 and determined when the user selects one or more than onefrom a plurality of preset images. Composite image generation portion 53outputs the composite image to transmission portion 55.

Transmission portion 55 transmits the composite image input fromcomposite image generation portion 53 to the picture phone of the personon the other end via wireless circuit 22.

Here, movement of the masked region will be described. FIG. 14 is afirst diagram showing one example of the masked region after movement.In this example, a case where the operator moves in the pick up image isshown while human eye is used as the feature. Masked region 110 shown inFIG. 7 moves as image 101 of the operator moves. In FIG. 14, region 110Ashown by hatching is the moved masked region. A region outside themasked region is newly set as a masked region.

FIG. 15 is a second diagram showing an example of the masked regionafter movement. In this example, the case where the pick up range ofcamera 24 is moved to the left is shown, while characters in the pick upimage are used as the feature. With regard to masked region 113 shown inFIG. 9, characters “□ □ Company” in the pick up image move as the pickup range of camera 24 changes. Masked region 113 shown in FIG. 9 ismoved by as much as the movement of characters “□ □ Company” in the pickup image to the position of masked region 113A shown by hatching in FIG.15.

FIG. 16 is a flowchart showing an example of a flow of incoming callprocessing. Incoming call processing is executed by control portion 21by executing the pick up image transmission program. Referring to FIG.16, control portion 21 judges whether or not an incoming call isdetected (step S01). When wireless circuit 22 receives an incoming callsignal, an incoming call is detected. A stand-by state is maintaineduntil an incoming call is detected (“NO” in step S01), and theprocessing proceeds to step S02 upon detection of an incoming call. Thatis, the incoming call processing is executed on condition that anincoming call is detected.

In step S02, whether or not a call answering instruction is accepted isjudged. Whether or not talk key 14B is pressed is judged. When a callanswering instruction is accepted, the processing proceeds to step S03,otherwise the processing is completed. This is because a communicationis not established if the operator has no intention to answer thecalling.

When the processing proceeds to step S03, a communication is establishedwith regard to the picture phone at the calling party. In step S03, avoice communication is commenced. Voice communication is now madepossible at this stage.

In step S04, whether or not the communication is by means of picturephone is judged. If the communication is by means of picture phone, theprocessing proceeds to step S05, and if not, the processing proceeds tostep S17. When the processing proceeds to step S05, and dynamic imagedata is received from the picture phone at the calling party, thedynamic image data is output to display control portion 30 and LCD 16 iscaused to display the dynamic image data. In step S17, whether or not acommunication by voice is completed is judged. The processing turns intoa stand-by state until communication is completed (“NO” in step S17),and upon completion of the communication, the processing is completed.Completion of the communication is judged by detection of communicationdisconnection or of pressing of communication completion key 14C.

In step S05, transmission of preset characters is started. Specifically,preset characters stored in advance in flash ROM 32 are read out andtransmitted to the picture phone on the other end via wireless circuit22. With these operations, the preset characters are displayed on thepicture phone at the other end. At this stage, an image obtained bypicking up by camera 24 is not transmitted to the picture phone at theother end.

In the next step S06, whether or not an image is transmitted is judged.Whether or not an image is transmitted may be set in advance, or may bejudged according to an instruction entered by the user through operationkey 14. When the image is transmitted, the processing proceeds to stepS07, and if not, it proceeds to step S13.

In step S07, whether to mask the image is judged. Whether to mask theimage may be judged according to setting that is set in advance, orjudged according to an instruction entered by the user. When judgment ismade according to an instruction entered by the user, camera 24 picks upan image of the operator and outputs the pick up image which is thendisplayed on LCD 16, and at the same time, a message asking whether tomask the pick up image is displayed on LCD 16 to prompt the user toenter an instruction. If a key of the operation keys 14 to whichindication of masking is assigned is entered by the user, the processingproceeds to step S08, and if not, it proceeds to step S14.

In step S08, masked region determining processing is executed. Themasked region determining processing, described later, is processing todetermine a masked region in the pick up image. In the next step S09,whether or not an instruction for image transmission is accepted isjudged. The masked region determining processing is repeated untilindication of a key of operation keys 14 to which entry of instructionfor image transmission is assigned in advance, and when the key isindicated, the processing proceeds to step S10. This is to allow thatafter the masked region is determined, the operator transmits theoperator's pick up image picked up and output firstly by camera 24.

In step S10, a composite image is generated. Specifically, such acomposite image is generated that of the pick up image output by camera24 after picking up the operator, the masked region determined in stepS08 is replaced by another image. Another image is a preset imagedetermined by the masked region determining processing in step S08 or animage painted out in a single color. In the next step S11, the compositeimage, instead of the preset characters that have started to betransmitted in step S05, is transmitted to the picture phone on theother end via wireless circuit 22. This allows that the composite imageis displayed on the picture phone on the other end.

In the next step S12, follow-up processing is executed. The follow-upprocessing, described later, is processing for moving the masked regionfollowing the movement of the feature in the pick up image. In the nextstep S13, whether or not the communication is completed is judged. Ifthe communication is completed, the processing is terminated, and ifnot, the processing goes back to step S10. This is for the purpose oftransmitting the composite image until the communication is completed.

In the meantime, in step S14, in a similar fashion to step S09, whetheror not an instruction for image transmission is accepted is judged. Theprocessing turns into a stand-by state (“NO” in step S14) until theinstruction for image transmission is accepted, and when the instructionfor image transmission is accepted, the processing proceeds to step S15.This is to allow that after the instruction, the operator transmits theoperator's pick up image picked up and output firstly by camera 24. Theprocessing may proceed immediately to step S15 without accepting theinstruction for image transmission.

In step S15, the pick up image is transmitted to the picture phone onthe other end via wireless circuit 22. This allows that the pick upimage is displayed on the picture phone on the other end. In the nextstep S16, whether or not the communication is completed is judged. Ifcompleted, the processing is terminated, and if not, the processing goesback to step S15. This is for the purpose of transmitting the pick upimage until the communication is completed.

FIG. 17 is a flowchart showing an example of a flow of masked regiondetermining processing. The masked region determining processing isprocessing executed in step S08 shown in FIG. 16. Referring to FIG. 17,the processing is branched according to the masked region designationmethod (step S51). The masked region designation method may be set inadvance, or may be selected by the user by entering an instruction tooperation portion 25.

When the masked region designation method is of masked regiondesignation, the processing proceeds to step S52; if non-masked regiondesignation, the processing proceeds to step S61; and if pattern imagedesignation, the processing proceeds to step S71.

In step S52, a position in the pick up image displayed on LCD 16 isaccepted. Specifically, a position corresponding to the positionindicated on touch panel 25A is identified, or a position of the pick upimage corresponding to the position of the pointer displayed on LCD 16and indicated by indication key 15 is identified. Then, the acceptedposition is set as the masked region (step S53). A predetermined rangefrom the accepted position may be set as the masked region, or a regiondefined by a plurality of positions accepted continuously may be set asthe masked region.

Then, whether or not designation of the masked region is completed isjudged (step S54). The designation of the masked region is judged to becompleted when the user's instruction to complete designation of themasked region is entered through operation key 14. When designation ofthe masked region is completed, the processing proceeds to step S55, andif not completed, the processing goes back to step S52. This is for thepurpose of enabling a plurality of masked regions to be designated.

In step S61, in a similar fashion to step S52, the position of the pickup image displayed on LCD 16 is accepted. Then, the accepted position isset as the non-masked region (step S62). A predetermined range from theaccepted position may set as the non-masked region, or a region definedby a plurality of positions accepted continuously may be set as thenon-masked region.

Then, whether or not designation of the non-masked region is completedis judged (step S63). The designation of non-masked region is judged tobe completed when the user's instruction to complete designation of themasked region is entered through operation key 14. When designation ofthe non-masked region is completed, the processing proceeds to step S64,and if not completed, the processing goes back to step S61. This is forthe purpose of enabling a plurality of non-masked regions to bedesignated. In step S64, a region of the pick up image other than thenon-masked region is set as the masked region, and the processingproceeds to step S55.

Further, in step S71, a pattern image stored in advance in flash ROM 32is read and displayed on LCD 16. Since the pick up image is displayed onLCD 16, an image in which the masked region included in the patternregion is superimposed onto the pick up image is displayed on LCD 16. Aplurality of pattern images may be stored in advance in flash ROM 32 sothat the user selects any one of the pattern images, or the user selectsone of the plurality of pattern images based on a feature extracted fromthe pick up image. Further, the pattern image may be enlarged or reducedbased on the size of the shooting object in the pick up image.

In step S72, whether or not an expansion instruction is entered isjudged. The expansion instruction is accepted when, after the positionof the masked region displayed on LCD 16 is accepted by touch panel 25Aor indication key 15, a position other than the masked region isaccepted continuously. When the expansion instruction is accepted, theprocessing proceeds to step S73, and if not, the processing proceeds tostep S74. In step S73, the masked region is expanded, and the processingproceeds to step S74. Specifically, a position that is other than themasked region and is accepted continuously after the position of themasked region displayed on LCD 16 is accepted by touch panel 25A ischanged to the masked region.

In step S74, whether or not a reduction instruction is entered isjudged. The reduction instruction is accepted when, after the positionof a region other than the masked region displayed on LCD 16 is acceptedby touch panel 25A, the position of the masked region is acceptedcontinuously. When the reduction instruction is accepted, the processingproceeds to step S75, and if not, the processing proceeds to step S76.In step S75, the masked region is reduced, and the processing proceedsto step S76. Specifically, the position of the masked region acceptedcontinuously after the position of the non-masked region displayed onLCD 16 is accepted by touch panel 25A is changed to the non-maskedregion.

In step S76, whether or not the masked region is determined is judged.In other words, whether or not expansion and reduction of the maskedregion is completed is judged. The masked region is judged to bedetermined when the user's instruction to determine the masked region isentered through operation key 14. When the masked region is determined,the processing proceeds to step S55, and if not, the processing goesback to step S72. This is for the purpose of enabling a plurality ofexpansion and reduction of the masked region.

In step S55, whether or not a preset image is selected is judged. Whenthe preset image stored in flash ROM 32 is selected, the processingproceeds to step S56, and if not selected, the processing proceeds tostep S57. In step S56, the selected preset image is set to the image ofthe masked region, and the processing goes back. This allows generationof a composite image in which the masked region of the pick up image isreplaced by the preset image.

Meanwhile, in step S57, a single color is set to the image of the maskedregion, and the processing goes back. This allows generation of acomposite image in which the masked region of the pick up image isreplaced by a single color image.

Masked region designation and non-masked region designation may becombined in designating the masked region. According to the designationmethod by masked region designation, a region in the pick up image thatis not desired to be transmitted can be designated. According to thedesignation method by non-masked region designation, a region in thepick up image that is desired to be transmitted can be designated.Accordingly, designation of the masked region can be made more simply bya combination of the designation methods.

FIG. 18 is a flowchart showing an example of a flow of follow-upprocessing. The follow-up processing is processing executed in step S12in FIG. 16. Referring to FIG. 18, a feature is extracted from the pickup image (step S81). The feature is represented by image of human eye,characters, or the like. Then, whether or not the feature is moved isjudged (step S82). Specifically, judgment is made as to whether there isany difference between the position of the currently extracted featureand the position of a feature extracted from a prior pick up image tothe pick up image from which the current feature is extracted. If thepositions of the features extracted from the two different pick upimages differ, it is judged that the feature has moved. When the featurehas moved, the processing proceeds to step S83, and if not moved, theprocessing goes back.

In step S83, the masked region is moved, and the processing goes back.The masked region is moved in parallel according to a vector obtainedfrom the two features. When the masked region before movement is incontact with either of the four sides of the pick up image, the regionbetween the contact side and the masked region is newly specified as amasked region.

FIG. 19 is a flowchart showing an example of a flow of callingprocessing. The calling processing is processing executed by controlportion 21 by executing the pick up image transmission program.Referring to FIG. 19, control portion 21 judges whether or not thepicture phone is started (step S21). The processing turns into astand-by state until the picture phone is started, and upon starting ofthe picture phone, the processing proceeds to step S22. That is, thecalling processing is processing executed on condition that the picturephone is started. In step S22, in similar fashion to step S07 in FIG.15, whether the image to be transmitted should be masked is judged. Whenthe image is masked, the processing proceeds to step S23, and if not,the processing proceeds to step S32.

In step S23, the masked region determining processing shown in FIG. 16is executed. In the next step S24, whether or not a calling instructionis accepted is judged. When pressing of call key 14B is detected, thecalling instruction is accepted and the processing proceeds to step S25,and if not, the processing goes back to step S23. This is to allow thatthe operator makes a calling after the masked region is determined andtransmits a composite image in which the operator's pick up image pickedup and output by camera 24 is masked.

In step S25, a calling is made and the processing proceeds to step S26.In step S26, whether or not the calling is answered is judged, and whenanswered, the processing proceeds to step S27, and if not, theprocessing is terminated.

In step S27, voice communication is commenced and the processingproceeds to step S28. In step S28, a composite image is generated.Specifically, such a composite image is generated that of the operator'spick up image picked up and output by camera 24, the masked regiondetermined in step S23 is replaced by another image. Another image is apreset image determined by the masked region determining processing instep S23 or an image painted out by a single color. In the next stepS29, the composite image is transmitted to the picture phone on theother end via wireless circuit 22. Thus, the composite image isdisplayed on picture phone on the other end.

In the next step S30, the follow-up processing shown in FIG. 16 isexecuted, and the processing proceeds to step S31. In step S31, whetheror not the communication is completed is judged. If the communication iscompleted, the processing is terminated, and if the communication is notcompleted, the processing goes back to step S28.

Meanwhile, in step S32, the processing turns into a stand-by state untila calling instruction is accepted. Upon acceptance of the callinginstruction, the processing proceeds to step S33. In step S33, a callingis made and the processing proceeds to step S34. In step S34, whether ornot the calling is answered is judged, and when answered, the processingproceeds to step S35, and if not, the processing is terminated.

In step S35, voice communication is commenced and the processingproceeds to step S36. In step S36, whether to deliver an image isjudged. When the image is delivered, the processing proceeds to stepS37, and if not, the processing proceeds to step S39.

In step S37, the pick up image is transmitted to the picture phone onthe other end via wireless circuit 22. Thus, the pick up image isdisplayed on the picture phone on the other end. In the next step S38,whether or not the communication is completed is judged. If thecommunication is completed, the processing is terminated, and if thecommunication is not completed, the processing goes back to step S37.This is for the purpose of transmitting the pick up image until thecommunication is completed. In step S38, whether or not thecommunication is completed is judged. The processing turns into astand-by state until the communication is completed, and upon completionof the communication, the processing is terminated.

As explained above, cellular phone 1 in the present embodiment functionsas the picture phone, and determines a masked region for masking atleast a part of the pick up image when at least a partial region of thebackground of a pick up image of the operator is designated by touchpanel 25A, and generates and transmits a composite image in which themasked region of the pick up image is converted to a preset image or asingle color image. Therefore, a region such as background of the pickup image in which the operator is picked up is not transmitted, and itbecomes possible that part of an image obtained by picking up isselected and is transmitted. For example, when the background includesan image which the operator does not desire to transmit, it is possibleto prevent the image from being transmitted.

Further, designation of the masked region is possible while the pick upimage is displayed on LCD 16, and therefore, any desired region can beselected from the pick up image, and a region which is not transmittedcan be selected with ease.

Further, it is possible to display on LCD 16 an image in which a patternimage including a predetermined masked region is superimposed onto apick up image, and to expand or reduce the masked region by manipulatingtouch panel 25A. Hence, the mask is displayed and the masked region ischanged based on the accepted position of the display surface, andaccordingly, a region which is not transmitted can be selected easilyfrom the pick up image.

Further, when a feature such as eyes or characters is extracted from thepick up image, and the position of the feature changes in the pick upimage, then the masked region in the pick up image is moved according tothe change, and therefore, transmission of any region in the pick upimage which is not desired to be transmitted can be prevented even whenthe picking up range of camera 24 is varied or the shooting objectmoves.

Further, since the pick up image is not transmitted unless a maskedregion is determined, it is possible to reliably prevent transmission ofany image which is not desired to be transmitted.

While transmission of images on the picture phone function has beenexplained in the present embodiment, it is not limited to the picturephone function, and the present embodiment is applicable to transmissionof images in the meeting function for transmission and receiving of acombination of characters, audio, and images as represented by, forexample, Hello Messenger service provided by KDDI.

While explanation has been given in the present embodiment usingcellular phone 1 as one example of the image transmission apparatus, itwill be readily appreciated that the present invention can also be takenas such an image transmission method for causing the image transmissionapparatus to execute processing shown in FIGS. 16 through 19, or as sucha pick up image transmission program for causing a computer to executeprocessing shown in FIGS. 16 through 19.

1. An image transmission apparatus comprising: an imaging portion tooutput a pick up image obtained by picking up a shooting object; aregion designation portion to accept designation of at least a partialregion of said pick up image; a masked region determining portion todetermine a masked region to mask at least a part of said pick up imagebased on said accepted region; a composite image generation portion togenerate a composite image in which said masked region of said pick upimage is converted to another image; and a transmission portion totransmit said composite image.
 2. The image transmission apparatusaccording to claim 1, wherein said masked region determining portiondetermines said accepted region or a region other than said region assaid masked region.
 3. The image transmission apparatus according toclaim 1, further comprising a display portion to display said pick upimage, wherein said region designation portion includes a positionacceptance portion to accept a position of a display surface of saiddisplay portion.
 4. The image transmission apparatus according to claim3, wherein: said display portion displays an image in which a patternimage including a predetermined masked region is superimposed onto saidpick up image; and said masked region determining portion includes achange portion to change said masked region included in said patternimage based on the position of said accepted display surface.
 5. Theimage transmission apparatus according to claim 4, wherein said changeportion comprises: an expansion portion to expand said masked region;and a reduction portion to reduce said masked region.
 6. The imagetransmission apparatus according to claim 3, further comprising: afacial region detection portion to detect a facial region from said pickup image; and a pattern image generation portion to generate a patternimage by replacing said detected facial region said masked region,wherein: said display portion displays an image in which said generatedpattern image is superimposed onto said pick up image; and said maskedregion determining portion includes a change portion to change saidmasked region included in said pattern image based on said acceptedposition of said display surface.
 7. The image transmission apparatusaccording to claim 1, further comprising: a feature extraction portionto extract a feature from said pick up image; and a masked regionmovement portion to move said masked region in said pick up imageaccording to a change in the position of said feature in said pick upimage.
 8. The image transmission apparatus according to claim 1, whereinsaid transmission portion transmits said composite image on conditionthat said masked region is determined.
 9. The image transmissionapparatus according to claim 1, further comprising a call detectionportion to detect an incoming call, wherein said transmission portiontransmits, upon acceptance of an instruction to answer the incomingcall, a predetermined image until said masked region is determined. 10.The image transmission apparatus according to claim 1, wherein saidtransmission portion transmits, upon acceptance of an instruction forcalling, a predetermined image until said masked region is determined.11. An image transmission method comprising the steps of: acquiring apick up image obtained by picking up a shooting object; accepting adesignation of at least a partial region of said pick up image;determining a masked region for masking at least a part of said pick upimage based on said accepted region; generating a composite image inwhich said masked region of said pick up image is converted to anotherimage; and transmitting said composite image.
 12. The image transmissionmethod according to claim 11, wherein said masked region determiningstep includes the step of determining said accepted region or a regionother than said region as said masked region.
 13. The image transmissionmethod according to claim 11, further comprising the step of displayingsaid pick up image, wherein said designation of said region acceptingstep includes the step of accepting a position in said displayed pick upimage.
 14. The image transmission method according to claim 13, wherein:said displaying step includes the step of displaying an image in which apattern image including a predetermined masked region is superimposedonto said pick up image, wherein; said masked region determining stepincludes the step of changing said masked region included in saidpattern image based on said accepted position in said pick up image. 15.The image transmission method according to claim 14, wherein saidchanging step includes the steps of: expanding said masked region; andreducing said masked region.
 16. The image transmission method accordingto claim 13, further comprising the steps of: detecting a facial regionfrom said pick up image; and generating a pattern image by replacingsaid facial detected region by said masked region, wherein: saiddisplaying step includes the step of displaying an image in which saidgenerated pattern image is superimposed onto said pick up image; andsaid masked region determining step includes the step of changing saidmasked region included in said pattern image based on said acceptedposition in said pick up image.
 17. The image transmission methodaccording to claim 11, further comprising the steps of: extracting afeature from said pick up image; and moving said masked region in saidpick up image according to a change in the position of said feature insaid pick up image.
 18. The image transmission method according to claim11, wherein said transmitting step includes the step of transmittingsaid composite image on condition that said masked region is determined.19. The image transmission method according to claim 11, furthercomprising the step of detecting an incoming call, wherein saidtransmitting step includes the step of transmitting a predeterminedimage, upon acceptance of an instruction to answer the incoming call,until said masked region is determined.
 20. The image transmissionmethod according to claim 11, wherein said transmitting step includesthe step of transmitting a predetermined image, upon acceptance of aninstruction for calling, until said masked region is determined.
 21. Animage transmission program product for causing a computer to execute thesteps of: acquiring a pick up image obtained by picking up a shootingobject; accepting designation of at least a partial region of said pickup image; determining a masked region for masking at least a part ofsaid pick up image based on accepted said region; generating a compositeimage in which said masked region of said pick up image is converted toanother image; and transmitting said composite image.